In many data processing applications—e.g., data integration or Extract, Transform, Load (ETL) processes—execution of jobs commonly involves use of parameters (or variables) as arguments for specific invocations of a job. Parameters includes, for example, frequently changing values such as a password for a database connection, or performance tuning values (e.g., “maximum number of threads”). Parameters can even include specific semantic variations in a job such as a “date range” parameter (e.g., “between (Jan. 1, 2001, Jan. 1, 2006)”). Generally the values of parameters (or variables) are changed depending upon various circumstances; for example, for daily runs of a particular job (e.g., a report for “date range”), or for rare occurrences (like changing the URL of a database, if the database has been moved). Typically, there are also programmatic interfaces that may be used to automatically set the values of some parameters, for example, “today's date”.
Typically, to manage the development and administration of complex data processing applications, the lifecycle of such data processing applications are loosely organized into phases. Examples of phases include a design (or development) phase (during which each component or job of an application is modeled or coded), a deployment preparation phase (during which the components of an application are uniquely identified and configured, compiled or built, the total set of database resources are identified, and so on), a packaging (or assembly) phase (during which an application installing package is assembled, including all code, shared libraries, and configuration files), a deployment (or install) phase (during which the application package is installed onto a data processing runtime environment, and resource references are mapped to live resources), an administration phase (during which deployed jobs may be configured or administered), and an execution instance phase (during which instances of deployed jobs are executed). During each phase, different users are generally involved—e.g., during the design phase, software programmers develop specific components or jobs, while during the administration phase, administrators typically deploy and monitor applications and jobs in the production runtime environment.
Due to the large number of parameters that are typically involved in large enterprise data processing applications, not every user may have a sufficient semantic understanding about each job and the parameters associated with each job, especially since there are usually multiple different users involved in the different stages of an application's development. For example, an administrator who executes a specific job may not be aware of the specific semantics associated with the setting of a particular parameter value. Thus, as a result, only the most common or well understood parameters (such as “password”) are the only parameters implemented within a job. Generally, there is also no easy way to determine the effect of a change in the value of a parameter at any point in the lifecycle of a job in a data processing application. For example, changing a parameter that was previously assigned the name of a particular database table could significantly affect an entire application. In general, there also is not a standard way to restrict when the value of a parameter can be safely changed.